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are designed to assess state government operations. From the 
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generally accepted government auditing standards. Those 
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sufficient, appropriate evidence to provide a reasonable basis for 
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believe that the evidence obtained provides a reasonable basis 
for our findings and conclusions based on our audit objectives. 
Members of the performance audit staff hold degrees in 
disciplines appropriate to the audit process. 
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The Legislative Audit Committee 
of the Montana State Legislature: 


This is our performance audit of Tourism Marketing and Promotion activities managed 
by the Department of Commerce. 


This report includes recommendations for improving the selection and evaluation of 
tourism grants, the research support and evaluation of marketing activities managed 
by regional tourism organizations across the state, and the overall documentation 
of tourism activities managed by the department. A written response from the 
Department of Commerce is included at the end of the report. 
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18P-03 REPORT SUMMARY 


In 2018, 12.4 million out-of-state visitors contributed $3.7 billion in travel 
spending to Montana’s economy. The Department of Commerce (department) 
is responsible for marketing and promoting the state to encourage 
nonresidents to visit the state and its communities, including managing a 
statewide marketing contract, administering tourism grants, and overseeing 
bed tax expenditures to regional tourism organizations. Our work found 
there are opportunities for the department to strengthen transparency and 


accountability regarding how they review and evaluate tourism grants, 


including the support and evaluation of bed tax expenditures relating to 
tourism promotion, which totaled over $8 million in fiscal year 2017. 


Context 


The department markets the state, administers 
a tourism grant program, and distributes 
Lodging Facility Use Tax resources— 
commonly referred to as the bed tax—collected 
from hotels, bed and breakfasts, guest ranches, 
resorts, and campgrounds to regional tourism 
organizations. These various tourism-related 
promotion activities are managed by the 
departments Montana Office of Tourism 
and Business Development. Audit work 
identified potential risks related to several 
tourism activities managed by the department, 
including how tourism grants are reviewed and 
selected, how a statewide marketing contract 
was selected by the department, and how the 
department oversees the expenditure of bed 
tax funds distributed to regional nonprofit 
tourism organizations. Consequently, as part 
of our work, we reviewed the selection process 
for tourism grants issued by the department in 
fiscal year 2017. We also reviewed the selection 
process for the statewide marketing contract 
in 2016 and the distribution and oversight 
of bed tax funds to those regional tourism 
organizations for the period of fiscal year 
2017. Currently, Montana is organized into six 
tourism regions, containing 19 Convention & 


Visitor Bureaus (CVB), which are regional 
nonprofit tourism organizations. 


Results 


We evaluated the department’s process 
for selecting a vendor to administer the 
state's marketing contract. We found the 
department followed best practices in the 
request-for-proposal process, _ including 
putting controls in place to mitigate potential 
conflicts of interest with department staff 
in the selection process. We also found 
the department actively manages the state 
marketing contract. There are current 
contractor performance concerns for the 
department to consider in the future when 
deciding if the contract should be extended. 
We also identified a lack of consistent support 
regarding how the department reviewed 
and awarded tourism grants, including how 
the results of those grants are evaluated. We 
noted there was limited justification for how 
the department determined which grants to 
award, with no documentation for several 
of those grants. In addition, final evaluative 
reports did not always indicate if the project 


(continued on back) 


was a success or met proposed goals. The lee ae 
department should develop and implement 


expectations to measure the impact of the 
Rees Partially Concur 

grants. Similarly, the department and the 
Tourism Advisory Council should improve Do Not Concur a 
oversight of the tourism regions and CVBs ; ; 

: Source: Agency audit response included in 
to clarify and enforce methodology research final report. 
and evaluation requirements for marketing 
methods funded by bed tax resources. The 
department should take steps to prioritize 
and clarify documentation requirements for 
issuing tourism grants and financial reporting 
processes. 


Audit recommendations address the need 
for the department to provide additional 
transparency and accountability for several 
tourism-related activities, including the need 
to strengthen documentation requirements. 
Recommendations include: 


¢ Developing and implementing proce- 
dures for its tourism grant program to 
document and support grant award 
decisions based on established grant 
award criteria, 


¢ Defining expectations to measurably 
evaluate the impact of tourism grants, 
including developing a structured 
template for grantees to use for final 
reporting, 


¢ Working in conjunction with the 
Tourism Advisory Council to improve 
its oversight of Tourism Regions 
and Convention & Visitor Bureaus 
to clarify and enforce methodology 
research and evaluation requirements, 
and 


¢ Prioritizing and clarifying documen- 
tation requirements for issuing tourism 
grants and required financial reporting 
processes for Tourism Regions and 
Convention & Visitor Bureaus. 


For a complete copy of the report (18P-03) or for further information, contact the 
Legislative Audit Division at 406-444-3122; e-mail to lad@mt.gov; or check the web site at 


https://leg.mt.gov/lad/audit-reports 
Report Fraud, Waste, and Abuse to the Legislative Auditor's FRAUD HOTLINE 


Call toll-free 1-800-222-4446, or e-mail LADHotline@mt.gov. 


Chapter | — Introduction and Background 


Introduction 


Travel and tourism-related activities provide economic growth opportunities for states. 
Many states, including Montana, direct resources towards public tourism promotion 
to attract out-of-state visitors and promote tourism growth. According to the Institute 
for Tourism and Recreation Research (ITRR) located at the University of Montana, 
12.4 million out-of-state visitors contributed $3.7 billion in travel spending to 
Montana’s economy in 2018. In Montana, the Department of Commerce (department) 
is responsible for marketing and promoting the state to encourage nonresidents to 
visit the state and its communities. The department markets the state, administers 
a tourism grant program, and distributes Lodging Facility Use Tax resources— 
commonly referred to as the bed tax—collected from hotels, bed and breakfasts, guest 
ranches, resorts, and campgrounds to regional tourism organizations. These various 
tourism-related promotion activities are managed by the department’s Montana Office 
of Tourism and Business Development (MOTBD). Based on legislative interest, the 
Legislative Audit Committee prioritized a performance audit of the effectiveness of 
department activities to promote Montana as a tourist destination. 


Audit Objectives, Scope, and Methodologies 


We developed the following audit objectives as part of our examination of tourism- 
related activities. 


¢ Does the department monitor the impact and effectiveness of its tourism 
marketing activities? 


¢ Does the department ensure the Regions and Convention & Visitor Bureaus 
expend state bed tax funds in accordance with statutory and department 
requirements, including measuring outcomes of regional tourism marketing 
efforts? 


During our audit assessment work, we identified potential risks related to several tourism 
activities managed by the department, including how tourism grants are reviewed and 
selected, how a statewide marketing contractor was selected by the department, and 
how the department oversees the expenditure of bed tax funds distributed to regional 
nonprofit tourism organizations. Consequently, as part of our work, we reviewed the 
selection process for tourism grants issued by the department in fiscal year 2017. We 
also reviewed the selection process for the statewide marketing contract in 2016 and 
the distribution and oversight of bed tax funds to those regional tourism organizations 
for the period of fiscal year 2017. Currently, Montana is organized into six tourism 
regions, which includes 19 Convention & Visitor Bureaus (CVB), which are regional 
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nonprofit tourism organizations. To accomplish our work, we completed the following 


methodologies. 


¢ 


¢ 


Reviewed relevant state laws, administrative rules, and department policies. 


Reviewed 78 tourism grant applications and 33 awards for fiscal year 2017 to 
evaluate how grants are selected and monitored. 


Reviewed department administration of the 2016 Request for Proposal 
(REP) for the state marketing contract to assess how the current contract was 
selected. 


Conducted interviews with the Department of Administration’s State 
Procurement Bureau to understand the procurement process used to select 
the state marketing contract. 


Reviewed the department’s administration of the state marketing contract to 
check for contract compliance. 


Interviewed the ITRR to gain an understanding of their role in researching 
the impact of tourism across Montana and providing visitation estimates and 
other metrics. 


Reviewed how the department evaluates the impact of tourism activities in 
the state. 


Interviewed tourism officials in other states to gauge best practices in state- 
sponsored tourism activities. 


Conducted data analysis examining the distribution of tourism grants, 
nonresident visitation, nonresident expenditures, and bed tax revenues. 


Interviewed department staff and Tourism Advisory Council members to 
gain an understanding of the oversight of state bed tax expenditures. 


Reviewed marketing plans and financial audits of regional tourism 
organizations for fiscal year 2017. 


Visited 13 regional tourism organizations across the state to learn more about 
how they operate and gain their perspective on state oversight activities. 


Surveyed all six tourism regions and 19 CVB regional tourism organizations 
to gauge their opinion of state marketing activities and oversight. 


Department of Commerce Role 


The Montana Department of Commerce works with statewide and local partners, 


from private industry and small businesses, to enhance and sustain economic 


prosperity in Montana. To carry out this mission, the department has three divisions: 
the Montana Office of Tourism and Business Development (MOTBD), Housing, and 
Community Development. MOTBD aims to support businesses through technical 


assistance, research, and access to grants and loans while promoting visitation and 


film development to maximize the economic impact of tourism. Its tourism marketing 
and promotion activities are outlined in §15-65-121(2), MCA, which calls for 
the department to use its share of the 4 percent Lodging Tax for tourism and film 


promotion. This law provides the department broad discretion in how it focuses on 
promoting and marketing Montana, which includes national marketing campaigns 
and distributing tourism grants across the state to spur promotion and tourism asset 
and infrastructure development at the local level. The department also oversees the 
distribution of bed tax funding to regional tourism organizations. In fiscal year 2017, 
MOTBD’s total budget from all funding sources was roughly $44.3 million. This 
includes nearly $17 million raised from the 4 percent bed tax for use in areas like the 
state marketing contract, tourism grants, and motion picture location promotion. In 
fiscal year 2017, MOTBD was allocated 62.85 FTE. However, tourism promotion is 
approximately half of what the division does, with business development activities also 
managed by the division. We did not review those activities as part of our audit work. 


State Marketing Contract 


The department contracts with an advertising agency to market and promote the 
state of Montana to nonresident visitors. This advertising agency is referred to as the 
state’s ‘agency of record’ and has responsibilities in public relations, social media, 
media planning and buying, and web marketing. For the remainder of this report 
we refer to the agency of record as a vendor. The vendor is to act as a partner with the 
department in developing and implementing comprehensive and integrated marketing 
programs that promote Montana as a year-round travel destination. The vendor was 
selected through a competitive RFP 


process. The performance of the Table 1 
Annual Tourism Contract Expenditures 


FY 2017-2019 


vendor is evaluated annually by the 
department, and the contract may 


be renewed annually for up to seven Fiscal Year Total Expenditures 
2017 $10,334,230 


2018 $10,080,144 
2019 $12,396,561 


years before another RFP is required. 


The department has contracted with 


the current vendor since August 2016. 


Source: Compiled by the Legislative Audit 
Division from department records. 


Table 1 outlines what the department 
has paid the vendor for tourism 


promotion activities each year since 


the inception of the contract. 


The above amounts represent cumulative reimbursement to the vendor for each fiscal 
year it has been contracted by the department. Annual spending varies based on the 
type and frequency of promotion activities, with a schedule of hourly rates established 
in the contract for each service. The nature of this spending is directly related to the 
marketing and promotion of Montana as a travel destination for nonresident visitors. 
This includes integrated seasonal marketing campaigns, such as promoting skiing 
destinations, and conducting analysis of market trends and campaign effectiveness. 
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Tourism Grants 


MOTBD also administers a tourism grant program that awards funds to projects to 
develop Montana’s tourism and recreation industry. Funds are awarded to projects 
catered to increasing nonresident visitation. ‘The program is funded through Montana’s 
current 4 percent bed tax. The grant program is not outlined specifically in statute 
or administrative rule. However, the department has broad authority in §15-65-121, 
MCA, to use bed tax funds for tourism promotion. From fiscal year 2008 through 
2017, the department has awarded over $6 million for 212 tourism grants across the 
state. The figure below illustrates the number of grants and dollar awards of tourism 
grants issued by the department from fiscal year 2008 through 2017. 


Figure 1 
The Department Has Awarded Over $6 Million for 212 Tourism Grants Across the State 
FY 2008-2017 


1,000,000 
900,000 
800,000 
700,000 
600,000 
500,000 
400,000 
300,000 
200,000 


100,000 


2011 2012 2013 2014 


mas Number of Grants Awarded = Value of Grants Awarded 


Source: Compiled by the Legislative Audit Division from department records. 


Tourism Advisory Council 


Established in state law, the Tourism Advisory Council (TAC) acts as an oversight for 
the department and MOTBD. The role of TAC is outlined in §2-15-1816, MCA. No 
less than 12 members are appointed in staggered three-year terms by the governor from 
Montana’s private sector travel industry, including 1 member from a tribal government 
and 1 tribal member from the private sector. Each of the six tourism regions are 
required to have representation as well. The council oversees distribution of funds to 
each region and CVBs, namely by granting budget approval for each organization’s 
marketing plan on how to spend bed tax funds over the course of a fiscal year. Regions 
and CVBs may not receive any bed tax funds until their marketing plans are approved 
by TAC. In fiscal year 2017, TAC approved 24 marketing plans worth over $8 million 
combined in bed tax funds. The expenditure of these funds is guided by a Regulations 


and Procedures department document for regions and CVBs and referenced in 
ARM 8.119.101. In addition, TAC provides general advice to the department related to 
tourism promotion. TAC also approves travel research carried out by the University of 
Montana’s ITRR. On an annual basis, ITRR receives 2.5 percent of the collected state 
bed tax for travel research purposes, with TAC prioritizing those research projects on 


an annual basis. 


Tourism Regions and Convention & Visitors Bureaus 


MOTBD is statutorily obligated to allocate a portion of bed tax revenues to certain 
qualifying regional and local nonprofit organizations dedicated to tourism marketing 
and promotion. These nonprofit organizations fall into one of two categories: tourism 
regions and CVBs. The number of regions in the state is currently set at six by TAC, 
although TAC may modify the tourism regions if it so desires. Tourism regions receive 
22.5 percent of bed tax revenue from their region of the state. However, this amount 
fluctuates depending on the number of CVBs within a region’s territory. There are 
currently 19 CVBs across the state, although this number may increase or decrease 
depending on which communities qualify to have a CVB based on criteria set in 
§15-65-121, MCA. Per state law, a nonprofit organization in a local area within the 
city limits, consolidated city-county, resort area, or resort area district may qualify 
to be a CVB if 22.5 percent of the amount of bed tax revenues in that area exceeds 
$35,000. Fifty percent of that revenue goes to the region in which the CVB resides. 
The remaining 50 percent in bed tax revenue goes to the CVB itself. A local area can 
have only one CVB, and it must have the backing of a local government authority, 
such as a city council or county commission, to be approved as a CVB. For the period 
of our review (fiscal year 2017), there were 18 CVBs around the state. However, the 
city of Havre qualified as a CVB in late 2017, bringing the total number of CVBs 
to 19. Figure 2 (see page 6) provides an illustration of the current six tourism regions 
and 19 CVBs located across Montana. 
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Figure 2 
Montana Tourism Regions and CVB Locations 


Hel ream | 


West Yellowstone +e = Convention and Visitors Bureau Office 


Glacier Country Central Montana 


=a Missouri River Country | Southwest Montana 
Yellowstone Country i Southeast Montana 


Source: Compiled by the Legislative Audit Division from department records. 


In fiscal year 2017, the six tourism regions and 18 CVBs operating at this time were 
awarded a total of over $8 million. Regions and CVBs must have their annual 
marketing plan approved by the TAC before receiving funds. Regions and CVBs work 
alone, together, and with the 


department in devising Table 2 ; ; 

a ‘ d Total Bed Tax Allocations to Tourism Regions for 
Hee | ee Approved Marketing Plans 
promotions for their specific FY 2017 
areas of the state. The following Region Bed Tax Allocation 
two tables illustrate the Central Montana $447,886 
allocation of bed tax funds to Glacier Country $1,650,000 
the six tourism regions and Missouri River Country $268,691 
18 CVBs active in fiscal year Southeast Montana $638,300 
2017. Southwest Montana $416,560 


Yellowstone Country $2,016,000 
Total $5,437,437 


Source: Compiled by the Legislative Audit Division 
from department records. 


Table 3 


Total Bed Tax Allocations to CVBs for Approved Marketing Plans 
FY 2017 


CVB Bed Tax Allocation 
Belgrade CVB $39,200 
Big Sky CVB $293,540 
Billings CVB $441,895 
Bozeman CVB $428,000 
Butte CVB $110,910 
Dillon CVB $25,800 
Gardiner CVB $45,739 
Glendive CVB $22,000 
Great Falls CVB $169,651 
Helena CVB $142,000 
Kalispell CVB $130,000 
Livingston CVB $26,999 
Miles City CVB $35,000 
Missoula CVB $296,713 
Red Lodge CVB $21,150 
Sidney CVB $26,900 
West Yellowstone CVB $387,836 
Whitefish CVB $137,703 
Total $2,781,036 


Source: Compiled by the Legislative Audit Division from department records. 


Figure 3 (see page 8) provides total bed tax revenue raised in each of the six tourism 
regions from fiscal year 2008 through 2017. As illustrated by the figure, bed tax revenues 
have remained fairly static in most regions, except for the Glacier and Yellowstone 
regions which have greater urban population centers and tourism attractions. 
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Figure 3 
Glacier and Yellowstone Tourism Regions Have the Highest and Have Seen the Most Increase in 4% 
Lodging Tax Revenue in Recent Years 
FY 2008-2017 


Yellowstone Country 


Glacier Country 


Southeast Montana 


Southwest Montana 


Central Montana 


Missouri River Country 


Millions 


2008 2009 2010 2011 2012 2013 2014 2016 2017 


Source: Compiled by the Legislative Audit Division from department records. 


Report Contents 


‘The remainder of this report includes chapters detailing our findings, conclusions, and 


recommendations in the following areas: 


¢ — Chapter II addresses the awarding and administration of the state marketing 
contract and improvements in the administration of tourism grants issued by 
the department. 


¢ Chapter II addresses the department’s oversight of tourism regions and 
CVBs, ways to improve the support and evaluation of regional marketing 
methods, and improving documentation of tourism activities administered 
by the department. 


Chapter II — State Marketing Contract 
and Tourism Grant Program 


Introduction 


Our first audit objective focused on determining if the Department of Commerce 
(department) monitors the impact and effectiveness of its tourism marketing activities. 
Our work focused on the awarding and administration of the state marketing contract 
and administration of the department’s Tourism Grant Program. As part of our 
work, we assessed whether the department appropriately awarded the state marketing 
contract in 2016 and if it monitors the state marketing contract activities. We also 
reviewed the processes by which the department selects and monitors tourism grants. 


Our work found the department followed appropriate procurement practices when it 
reviewed and awarded the state marketing contract in 2016. We also determined the 
department actively manages the contract. We determined the department conducts 
marketing research and analysis in accordance with industry standards to measure 
the effectiveness of its tourism marketing activities. However, we identified areas 
for improvement in how the department selects and monitors the tourism grants it 
issues. Our review of tourism grants found a lack of consistent support for awarding 
grants, as well as a need to improve evaluation of the impact of completed grants. This 
chapter discusses our conclusions, findings, and recommendations related to the state 
marketing contract and tourism grants issued by the department. 


State Marketing Contract Award Process 
Received Public Scrutiny 


When the state marketing contract was awarded to the current vendor in 2016, there 
was considerable media attention given to the process. Concerns raised included how 
the department selected an out-of-state vendor and the perception that there were 
conflicts of interest with department staff and their evaluation process when reviewing 
and approving the vendor. Because of these concerns, we reviewed the request-for- 
proposal (RFP) process used by the department to review and award the current 


marketing contract. 


Department Established a State Marketing 
Contract RFP Review Process 


At the onset of the 2016 state marketing contract RFP, the department issued a 
guidance to prospective contractors explaining its evaluation process. This included a 
scoring guide, weighted evaluation criteria, cost proposal formula, and a description of 
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the RFP process for any interested respondents. The evaluation process was split into 
three tiers worth a combined 10,000 points. Each is described below: 
¢ Tier one was worth a total of 5,500 points and consisted of the respondent’s 
response to the RFP, outlining its profile, experience, and qualifications 
for the contract. Evaluation criteria consisted of five categories and 
35 sub-categories. If the respondent did not receive at least 75 percent of 
the available 5,500 points in tier one (4,125 points), it was eliminated from 
further consideration. 


¢ Tier two review was worth 2,000 points and consisted of a cost proposal 
formula for each respondent, with 14 weighted cost categories. Maximum 
points per category were awarded to the respondent with the lowest cost 
proposal, with other proposals receiving a percentage of the points available 
based on their cost relationship to the lowest proposal in that category. After 
tier two's conclusion, the department would invite up to five of the highest- 
scoring respondents remaining to deliver a maximum two-hour presentation 
as tier three of the process, so long as they were within 1,250 points of the 
highest scoring respondent. If no other respondent was within 1,250 points 
of the highest scoring respondent at that point, the department retained the 
right to award the contract to the highest scoring respondent. 


¢ The tier three review consisted of respondent presentations for the five 
highest-scoring respondents from the tier two review. This tier was worth 
2,500 points over seven different categories. After the three-tier process, 
whichever responding applicant had the most points would be awarded the 
contract. 


We examined the RFP process used to award the state marketing contract. We 
reviewed the proposals of all 11 respondents, including cost proposals, and compared 
the consensus scores each received from the proposal review team to the established 
scoring criteria. We also reviewed the presentations of the five respondents selected as 
part of the tier three review and again compared the consensus scores each presentation 
received to the established scoring criteria. 


Steps Were Taken to Address Potential Conflict of Interest 


We also examined the steps the department took throughout the procurement process 
leading up to the awarding of the state marketing contract. Before reviewing the 
proposals, each member of the department’s RFP review team submitted attestations 
on whether they had any potential conflicts of interest. We noted a member of the 
department's RFP review team attested to having a conflict of interest and disclosed 
they had a relative who worked for one of the applicants. Upon review, the Department 
of Administration State Procurement Bureau determined the conflict of interest was 
minor since the relative was not directly involved in the proposal. It was determined 
the RFP review team member could remain involved in the process so long as several 
conditions were met. These conditions included adding an additional member to the 


department's RFP review team and including an employee from the State Procurement 
Bureau in all meetings held by the team. The review team consisted of four department 
management staff and one staff member from state procurement. These conditions 
were implemented and allowed the department to ensure a control was in place to 
mitigate any perceived conflicts of interest. 


Vendor Review Scores Were Supported 


The department received 11 proposals in response to its 2016 state marketing contract 
RFP. We reviewed all 11 of these proposals and compared the consensus scores each 
proposal received from the department to the criteria established in its scoring matrices 
based off the RFP itself: We found support for the review process and the scores each 
proposal received from the department’s review team. Per the conditions laid out in the 
REP, the top five scoring proposals were invited to deliver an in-person presentation 
of their proposals to the department. We reviewed each presentation for content and 
substance, again comparing it to established scoring matrices. We found documented 
support for the scores each presentation received from the RFP review team. As per 
the process established in the RFP, the respondent with the highest combined proposal 
and presentation score was awarded the state marketing contract. 


The Department Actively Manages the Current Marketing 
Contract, Including Performance Improvements 


We reviewed how the department manages the current marketing contract to ensure 
promotion of tourism activities meets contractual requirements. The vendor is contracted 
to deliver integrated marketing and promotion services for the state of Montana. The 
department reimburses the vendor several million dollars a year for these services. We 
noted the department conducts an annual review process by which a department review 
committee grades the vendor on 37 performance marks and standards. In our review 
of the annual review performed in May 2018, we noted numerous concerns identified 
by the review committee regarding the overall performance of the vendor in meeting 
contract expectations. ‘The consensus of this committee was the contracted vendor was 
“passing” 19 benchmarks, “needing improvement” on 13 benchmarks and “failing” 
4 benchmarks. One benchmark was graded as not applicable, as the department had 
not requested the contractor provide website development services as outlined in the 
original contract, with these services being fulfilled by another vendor. A summary of 
some of the improvements from the contract review committee are noted below. 
¢ = Unfamiliarity with Montana both geographically and culturally, leading to a 
disconnect between the department’s expectations and some of the services/ 
products delivered. For example, department staff noted that the contractor 


creative team was unprepared to answer questions on how concepts integrate 
with media placement. 
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¢ Struggle to use the established Montana brand. Department staff indicated 
there was a lack of deep understanding on the part of the contractor of 
Montana’s products and experiences, with a lack of familiarity with the state. 


¢ Poor communication, including multiple requests for updates by the 
department before hearing back. Because of a lack of communication, there 
were concerns the contractor represents themselves as uneducated with the 
state and its products, towns, and partners. 


¢ Rushed timelines and projects leading to additional expenses. Department 
staff identified a lack of understanding of creative requests, where multiple 
rounds of concepts were required, resulting in additional time and expenses. 


Because of these shortcomings, the contract review committee opted to place the 
vendor on a probationary status, with a corrective action plan jointly developed 
between the department and the vendor to address the deficiencies identified. The 
committee recommended the department renew the contract but require the vendor to 
meet defined deliverables or face termination of the contract. 


The department followed up with a mid-year review of contract expectations in 
December 2018. Department staff and our review of documentation indicated the 
vendor made improvements in some areas, such as more closely aligning marketing 
efforts with the Montana brand and values. Other areas such as the need to have a 
senior staff member for the contractor be in Montana were still underway. Based on the 
December 2018 review, department staff indicated having the contractor place a senior 
team member in the Helena office was a priority, as this has led to a gap in creative input 
from the contractor. Most recently, the department conducted a contract performance 
review in May 2019 and decided to renew the contract for another year. The result of 
this review indicated the vendor has continued to make improvements in the areas 
previously identified as deficiencies, with the department’s review highlighting many 
positive aspects of the ongoing work of the vendor. The annual review also indicated 
there are still several areas in need of improvement, such as the need to more forcefully 
leverage social media opportunities to promote tourism in Montana. According to 
department staff, they are currently in the process of developing a new plan of action 
to address ongoing areas of improvement. While our review indicated the department 
has taken active steps to manage the marketing contract, their annual reviews have 
identified ongoing areas for improvement regarding the performance of the contractor. 
It will be important for the department to continue to manage the contract in a way 
that ensures the effective promotion and marketing of tourism-related activities in the 
state. If contract issues continue, the department may need to consider reissuing the 
contract in the future. 


a 
CoNcLUSION 


The department followed procurement best practices for the selection for 

the current state marketing contract. The department actively manages the 
contract and has identified needed improvements regarding the performance 
of the contracted vendor. If issues continue, the department may need to 
consider reissuing the contract in the future. 


The Department Conducts Research and 
Analysis by Industry Standards 


State destination marketing organizations, like the department, rely on accurate 
information to make informed decisions. This helps them decide where to best invest 
limited marketing resources and determine if past investments were worthwhile. 
The department contracts with a private sector vendor, separate from the marketing 
contract vendor, to conduct studies of past marketing campaigns. Studies conducted 
include calculating the number of incremental visitors from various types of marketing 
campaigns. Incremental visitors are the number of visitors which visited Montana 
specifically because of a marketing campaign. Conversely, these same studies identify 
the type of visitor most likely to be interested in traveling to Montana and what kind 
of media they typically consume. For example, the department uses these studies to 
direct the investment of marketing resources by targeting high return demographics. 
In addition to these studies, the department uses digital methods for tracking 
effectiveness of marketing campaigns. These methods include using commercially 
available programs which allow for tracking of mobile phones which have viewed a 
travel advertisement for Montana or spent time on the state’s travel website. If the end 
user decides to visit Montana, their cell phone data will show when they make the 
trip. Similarly, commercially-available information from major credit card companies 
also allows the department to track when and where nonresident visitors are making 
purchases in the state. 


Another resource the department uses is the Institute for Tourism & Recreation 
Research (ITRR), based out of the University of Montana in Missoula. ITRR conducts 
travel and recreation research in Montana, including statewide nonresident travel 
surveys estimating nonresident visitation numbers, how much travelers spend while 
visiting the state, and several other metrics. [TRR also makes an annual estimate of the 
economic contribution of nonresident travelers to Montana’s economy. The department 
uses this publicly-available information as well, to help understand how past marketing 
campaigns fared or to identify other factors that have affected the tourism industry in 
Montana. As part of our work, we examined bed tax revenues relative to nonresident 
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visitor spending. One factor commonly examined is the relationship between the 
amount of bed tax revenues and the amount of nonresident visitor spending. For 
example, the bed tax revenue for the Yellowstone Country tourism region in calendar 
year 2017 was approximately $11 million, while nonresident travel spending for the 
same period was approximately $917 million. According to department staff, there are 
many factors that likely influence an individual’s decision to travel to Montana, with 
department promotion efforts playing a role. While this comparison illustrates there 
is an expected relationship between nonresident visitor spending and bed tax revenue, 
with bed tax revenue one potential indicator of nonresident visitor spending, there are 
additional factors which may influence an individual’s decision to visit Montana. For 
example, an individual may be influenced by friends or family or become aware of 
Montana as a destination through other media means rather than those directed by the 
department or by tourism regions or CVBs. Figure 4 (see page 15) illustrates the bed 
tax revenue generated in each of the state’s six tourism regions and the expenditures of 
nonresident visitors in those regions for calendar year 2017. As illustrated by the figure, 
nonresident expenditures in all cases exceeds the amount of bed tax revenue from that 
region. As expected, our work indicated that nonresident expenditures typically exceed 
bed tax revenues. 
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We also reached out to other states to learn how they conduct market research and 
analysis. We spoke to state marketing officials in Idaho, South Dakota, Utah, and 
Wyoming. We found all states had similar or identical methods for conducting research 
and analysis of their marketing efforts as Montana. For example, officials in Utah also 
contract with an outside entity to evaluate the effectiveness of marketing campaigns, 
with that entity conducting surveys before and after marketing campaigns to gauge 
the effectiveness of these various efforts. All states contracted with an outside vendor to 
assist in research and analysis, with all states embracing digital tracking abilities. 


ea 
CONCLUSION 


The department conducts market research and analysis common by industry 
standards to gauge the effectiveness of tourism promotion activities, including 
employing conversion studies, digital tracking, and other evaluative activities. 
Department research and analysis activities help ensure tourism promotion 
efforts are effective and bed tax expenditures are impacttul. 


Tourism Grants 


In fiscal year 2017, the department received 78 applicants and awarded 33 tourism 
grants totaling $949,500. To apply for a grant, interested organizations submit a 
grant application via an online program, answering a series of questions to establish 
their eligibility and the merits of their proposed project. These applications are given 
an initial review by department staff, largely to determine eligibility as established 
in the department’s tourism grant policy. This initial review team consisted of the 
grants manager and assistant grants manager. Eligible organizations include registered 
nonprofit 501(¢) organizations, tribal governments, and city or county governments. 
For example, in fiscal year 2017, Montana organizations which applied for grants 
included local chambers of commerce, cities, and various community organizations, 
such as local theaters and museums. These initial reviews also score each applicant 
on a scale of -4 to 4. Each reviewer also gives a brief written rationale for how they 
scored an application. All applications are forwarded to a final review team, regardless 
of eligibility or initial score. The final review team consisted of five management staff 
from the department. According to department staff, all applications are forwarded 
because the final review team needs to be prepared to let denied applicants know why 
their applications were not selected for a grant. Per department staff, the final review 
team decides which applications to propose awarding a tourism grant, with the final 
approval ultimately given by the department director. Each member of the final review 
committee rates each application on a pass/fail basis. The final review committee then 


meets to reach consensus on which applications to award a grant. Figure 5 illustrates 
the distribution and value of tourism grants awarded by the department in fiscal year 
2017 by county to each of the state’s six tourism regions. 


Figure 5 
Number and Value of Tourism Grants by County 
FY 2017 
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Source: Compiled by the Legislative Audit Division from department records. 


Tourism Grants Are Awarded Without 
Documented Justification or Rationale 


Our work identified several weaknesses in how tourism grants are reviewed and 
awarded to recipients. We reviewed tourism grants awarded by the department during 
fiscal year 2017 since this was the most recently completed grant cycle at the time of 
our audit. We noted decisions made by the final review team were not documented. 
While department staff maintain grants are awarded based on the established criteria 
in policy, department staff were unable to provide documentation supporting this. We 
tracked which grant applications were approved and denied compared to the initial 
review scores each application received. 
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There were a total of 78 tourism grant applications received by the department for fiscal 
year 2017 and we reviewed initial scores for 69 of these applications. For the other 
9 applicants, initial scores were not documented. According to department staff, three 
grants were part of a pilot program directed at tourism trade show assistance. However, 
there was no guidance or documentation available from the department regarding 
how these grants were reviewed and awarded. These three grants will be discussed 
in the following chapter. The three grants totaled nearly $7,000. For six grants, 
although the department was able to provide us with application materials, there was 
no documentation of the initial review process. It was unclear why these grants were 
passed along for final review and approved by the department. These six grants totaled 
$211,564. The following table provides a listing of those six grants, including the name 
of the organization, the city in which the organization is located, the purpose of the 
grant, and the amount of the grant. 


Table 4 
Examples of Unsupported Awarded Tourism Grants 
FY 2017 


Organization City Purpose Amount 


Range Riders Museum Miles City Repairing walls, insulating, new outlets $40,000 
and lighting, restroom plumbing, and paint 
and carpet. 


The Roxy Theatre Missoula Front ticket booth window installation, $67,605 
renewing the facade, and installing a 
replica neon marquee. 


Montana Dude Rancher’s Helena Website redesign to increase functionality. $3,766 
Association 


Ninepipes Museum of Early Charlo Installation of a new heating and air $28,338 
Montana conditioning system. 


Wheatland County Harlowton Upgrade existing electrical system and $40,666 
Chamber functionality for camping. 


Blackfoot Pathways Lincoln Purchase and installation of wheelchair $31,189 
accessible toilets, development of 
wheelchair accessible walkways, and 
way-finding signage. 


Source: Compiled by the Legislative Audit Division from department records. 


As discussed, we reviewed the initial review scores for the 69 grant applications for 
which initial scores were available. We determined the initial review score was not the 
only factor in determining which grants were awarded funding. Table 5 (see page 19) 
illustrates the initial review team scores for the 69 grants we reviewed as a part of our 
work. ‘The table is comprised of three rows, the first of which is the possible initial 
review team score, the second row is the number of applications which received that 
score, and the third row is the number of grants ultimately awarded with those scores. 


For the 33 grants awarded for fiscal year 2017, the below table includes the 24 awarded 
grants for which an initial review score was available. While awarded, the 9 grants 
noted above did not have initial scores available to review. Table 5 illustrates that it is 
unclear what the relationship is between the initial review score and why a grant was 
ultimately awarded. 


Table 5 
Available Initial Review Scores for Tourism Grants 


Possible Initial 


Review Score = 


Number of 
Applicants 
Received Score 


Number of 
Awarded Grants 
Received Score 


Source: Compiled by the Legislative Audit Division from department records. 


As can be seen in the above table, five grants received a zero or below as part of the 
initial review but were still ultimately awarded by the department. For example, one 
of the grants which was awarded received an initial review score of -3 with reviewer 
comments stating that the applicant lacked the appropriate cash match. Nonetheless, 
the applicant was ultimately awarded nearly $20,000. For the five grants applicants 
that received an initial review score of zero or less, total funding awarded to these 
grants totaled approximately $50,000. Conversely, there were three grant applications 
with a positive initial review score which were not awarded by the department. While 
there may be reasons for differences between the initial review scores relative to grant 
awards, the department was unable to provide any documentation as to why some 
grants were awarded and others were not. The initial review score does not appear to 
be the only factor in awarding grants. Without documentation, the rationale behind 
these decisions is unknown. Overall, our audit calls into question the credibility of 
how tourism grants are awarded, with the process lacking transparency. 


Other State Practices and State Policy Calls 
for Documenting Grant Awards 


We reviewed other states’ tourism grant review and award processes. We found other 
states with grant programs like Montana’s have clear criteria for scoring applications 
and documenting and supporting decisions for their awards process. For example, Utah 
has a similar grant program as Montana’s, but it has well-defined grant application 
evaluation standards. Utah has established a point-based scoring system with several 
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weighted categories such as clearly defined project goals and objectives as part of its 
tourism grant evaluation process. Each grant applicant must meet a minimum average 
score from the review committee members to be considered for funding, providing 
clear parameters for how grants are awarded. In Utah, out of a possible 100 points, 
an applicant must receive a score of 70 points or higher from a majority of the review 
committee members to be considered eligible to receive funding. In addition to other 
states, state policy in Montana relating to opening and awarding solicitation responses 
require evaluation committees keep meeting minutes and a master-scoring sheet 


documenting total scores for each proposal. 


The Department Lacks Procedures for 
Documenting Awarded Tourism Grants 


The department currently does not have procedures outlining expectations for 
reviewing and awarding tourism grants, including procedures for how tourism grant 
award decisions should be documented to support award decisions. During our audit, 
the department began drafting a procedure for documenting tourism grants award 
decisions. The department has not yet implemented the process at this time. This 
draft procedure calls for the final review team to keep notes to document why certain 
applications are awarded grants and others are not, including applications be evaluated 
based. on the established criteria with documentation of how awarded grants meet 
those criteria. Department staff indicated this new procedure will be implemented for 
the next cycle of tourism grants. 


Ne 


RECOMMENDATION #1 


We recommend the Department of the Commerce develop and implement 
procedures to document and support grant award and nonaward decisions for 
tourism grants based on established grant award criteria. 


To 


Measurement of Tourism Grant Impacts Can Be Improved 


Each tourism grantee is required by contract to submit a final report to the department 
within one year of their project’s completion and before September 1 of the subsequent 
fiscal year. However, the contract and current department guidance does not provide 
details on what the contents of a final report should include or what constitutes a 
project’s success. Our review of all 33 tourism grants awarded in fiscal year 2017 found 
wide variations in the quality and substance of final reports. Several reports did not 
provide any indication of project outcome, and many others provided little in the way 


of details. Our work found 9 of 33 (27 percent) of the final reports did not provide 
clear indication of whether the project was a success. For example, one grantee with a 
project related to increasing museum attendance submitted a final report only noting 
the project had been completed, with no indication of what attendance goals were set 
or what would constitute success. This grant totaled $40,000. Similarly, another project 
consisted of rebuilding a community grandstand structure, with the goal of increasing 
local rodeo attendance. This grant totaled $15,500. While the final report indicated 
the rebuilding effort was a success, there was no information submitted regarding what 
attendance goals were or if those goals were met. The total amount of funding awarded 
to these nine grants was $393,486. 


Content and Use of Final Reports Is Not Standardized 


The department does not have an internal grant administration procedure providing 
guidance to grantees on how the impact of grants should be evaluated or the 
information these reports should contain. Consequently, the quality of these final 
reports varied and provided limited use in assessing how grant monies were being used. 
We determined the final reports represented more of a formality than a substantive 
part of the grant program, providing limited accountability for the use of these public 
funds. For example, according to department staff, final reports are generally used 
as anecdotal examples for future prospective applicants to use as a model for similar 
projects instead of being used to evaluate if a project achieved its goals or had its 
intended impact. Department staff indicated these final reports were frequently used 
as part of media material generally for the purposes of program outreach. Final reports 
are commonly used in press releases to anecdotally highlight the success of the grant 
program. ‘The final reports are not used by the department as benchmarks for future 
grant applications or to measure the impact of these public funds. 


Other States Offer Best Practices for Grant Evaluation 


We contacted other states in the region to discuss their processes to evaluate tourism 
grant programs and how these processes compare to the department’s. We found 
other states had more defined processes to evaluate their tourism grant programs. For 
example, we noted the Utah Office of Tourism uses a structured template outlining 
what is required of grantees as part of a final evaluation report. Prospective grantees are 
required to estimate the projected impact and returns their grant will have and then 
use the structured template to record the actual impact and returns of their project. 
This requires tourism grant recipients to more fully evaluate the success and impact 
of their tourism grants. In this manner, awarded tourism grants in Utah require 
more accountability for the issuance of public funds to support marketing efforts for 
tourism-related organizations. 
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Impacts of Tourism Grants Should Be Measured 


While the department has wide discretion in how it spends bed tax dollars in support 
of promoting tourism in the state, it is critical these limited funds be invested in a way 
which spurs additional visitation across the state. The department’s tourism grant efforts 
represent an opportunity to help local organizations and communities increase visitation 
to their areas. It is important the success of awarded tourism grants be measured to 
ensure meaningful use of bed tax funds. More robust efforts to evaluate the impact 
of these public funds could provide important information to key stakeholders, such 
as legislators and tourism-industry officials, including what types of projects have the 
most impact on promoting tourism growth in Montana. ‘The department could use this 
information to inform future grant applicants and incorporate evaluation expectations 
into the award process for issuing tourism grants. By not requiring grantees to submit 
a structured evaluation of the impact of an awarded grant, the department is unable 
to easily determine which grants are most successful and which are less successful. 
While we recognize not all grant activities may lend themselves to be easily evaluated, 
a lack of a structured evaluation process represents a missed opportunity in educating 
future grant applicants or guiding grant applicants on how to measurably evaluate the 
impact of tourism grants. This raises the possibility of grants being approved which 
have a lesser impact promoting and marketing a Montana community as a tourism 


destination. 


Ne 


RECOMMENDATION #2 


We recommend the Department of Commerce develop and implement: 


A. A process with defined criteria to measurably evaluate the impact of 
tourism grants or document why an individual grant’s activities are not 
measurable, and 


B. Clear guidelines for grantees to follow and a structured template to use 
as part of a final reporting process. 


Chapter III — Regional Tourism 
Organization Oversight and 
Overall Documentation 


Introduction 


Our second objective focused on determining if the Department of Commerce 
(department) ensures the six regional tourism organizations and 19 Convention & 
Visitor Bureaus (CVB) across Montana expend state bed tax funds in accordance with 
statutory and department requirements. This includes measuring outcomes of regional 
tourism marketing efforts. Our work found the department conducts financial audits 
on an annual basis to ensure regions and CVBs meet the legal spending requirements 
of applicable rules and regulations. While this was positive, our work also determined 
regions and CVBs do not consistently support their various marketing methods with 
required research into the basis upon which the marketing method was selected. 
We also found the department does not consistently require regions and CVBs 
to evaluate the impacts of marketing methods used to promote tourism in their 
region of the state. In addition, over the course of the audit we observed a pattern 
of inconsistent documentation related to tourism grants and for financial reporting 
processes in the department for region and CVB activities. This chapter discusses our 
findings, conclusions, and recommendations related to region and CVB oversight and 
documentation issues within the department. 


Field Interviews and a Survey Helped 
Gauge Region and CVB Opinions 


As part of our work for this objective, we gained insight from regions and CVBs via field 
interviews with several of these organizations and a survey sent to every organization. 
We visited 13 of the 25 tourism organizations, including each of the six regions and 
at least one CVB from each region. We conducted interviews with small, medium, 
and large organizations across the state, since the nature of the challenges facing these 
organizations varies in part by their size and geographic location. We discussed the 
challenges these organizations face and how they view working with the department 
and the Tourism Advisory Council (TAC) in promoting tourism across Montana. 
‘These interviews were instrumental in gaining the perspective of some key department 
stakeholders. Some common themes emerged from these discussions. Overall, many 
believe the way bed taxes are allocated is fair, although more funding would be 
welcome. Several also believed certain aspects of the marketing requirements could 
be improved or modernized. Many organizations find the evaluation requirements 
in the department’s regulations and procedures to be either too cumbersome or not 
clear enough to follow. Some viewed evaluation reports as more of a formality than 
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a genuine exercise in evaluating marketing method effectiveness. We supplemented 
our on-site interviews with a survey to all 25 regional tourism organizations. With 
an overall response rate of 68 percent, survey respondents were generally positive 
regarding their interactions with the department and TAC. However, we received 
further confirmation that the evaluation requirements in the department’s regulations 
and procedures are unclear, with regional tourism staff seeking more guidance on how 
to meaningfully evaluate the marketing methods used to promote tourism in their 


regions. 


The Department Ensures Regions and 
CVBs Follow Spending Regulations 


The department’s regulations and procedures are comprised of the statutory basis 
and TAC-approved procedures which outline various spending requirements and 
restrictions of bed tax funds for regions and CVBs. ‘These regulations and procedures 
are incorporated by reference into ARM 8.119.101. This includes what is and is not 
allowed as administrative expenses, limits on administrative expenses as a percentage 
of spending, eligible and ineligible marketing expenditures, bidding and request for 
proposal processes, and overall fiscal management and bookkeeping responsibilities. 
For example, regions and CVBs may not expend more than 20 percent of bed tax 
funds on administrative expenses. To ensure all these requirements are being met, 
the department conducts an annual financial audit of each region and CVB. Our 
review determined that these financial audits are comprised of a review of every 
financial transaction the audited organization made over the past fiscal year. If an 
organization had any nonallowed expenses, they are required to recapture the bed tax 
funds. We reviewed the department's fiscal year 2017 financial audits of all 24 regions 
and CVBs in existence at that time and found the audits to be well-documented and 
all identified issues resolved. Examples of issues identified by these audits included 
items such as missing invoices or expenses not properly entered into the department's 
online reporting platform. At the end of each audit, department staff issue a final 
report indicating if the concerns had been resolved. Overall, we found these audits 
help ensure tourism regions and CVBs meet the legal spending requirements tied to 
bed tax funds they receive and are spending bed tax funds within the boundaries 


defined by state law and TAC. 


Region and CVB Marketing Plans Lack Complete Research 


Regions and CVBs are subject to oversight from the department and TAC. These 
requirements are outlined in a document titled “Regulations and Procedures for 
Regions/CVB Tourism Organizations Use of Lodging Facility Use Tax Revenue.” 
TAC’s authority is outlined in §2-15-1816, MCA. The department’s responsibilities are 
outlined in the regulations and procedures themselves, which are approved by TAC. 


Regions and CVBs are required to submit annual marketing plans to TAC for approval 
before receiving and spending any bed tax funds, as required in the regulations, 
procedures, and §15-65-122, MCA. Each marketing plan is comprised of individual 
marketing methodologies, which outline specific spending activities and goals for the 
upcoming fiscal year. Each marketing methodology is required to include research to 
support its inclusion as a worthwhile and meaningful investment of bed tax funds in 


tourism marketing and promotion in that region. 


We reviewed marketing plans for the 24 regions and CVBs in existence in fiscal year 
2017. Our work identified several regions and CVBs that did not include research for 
all their marketing methodologies as part of their online application and reporting 
requirements. We found nearly 10 percent of the total marketing methods employed 
by regions and CVBs in fiscal year 2017 lacked supporting research. For example, 
one organization proposed tourism-related website updates totaling $12,000 with no 
indication of how the improvements would impact the organization or how success 
would ultimately be measured. Out of a total of 273 marketing methodologies reviewed, 
we identified 25 marketing methodologies with no supporting research. The total value 
of these unsupported marketing methods was approximately $400,000. If marketing 
methods are not supported through research as required by the department’s regulations 
and procedures, there is limited accountability regarding the use of these public funds. 
We also identified instances of more common methodologies not being researched as 
well, such as print advertising and press promotions. Per department regulations and 
procedures, TAC is supposed to take into consideration if a methodology is supported 
by market research or previous experience to be deemed viable. However, based on our 


review work, this is not consistently occurring. 


Region and CVB Evaluation Reports 
Are Not Consistently Reviewed 


Per the department's regulations and procedures, regions and CVBs are also required 
to submit evaluation reports for each of the marketing methods within their marketing 
plans before January 1 of the following fiscal year. Evaluation reports explain whether 
a marketing method used by the region or CVB was successful or not. Evaluation 
reports should use evaluation techniques which are measurable and quantifiable. 
However, not all marketing methods are easily measurable and require other indicators 
to track success. 


We found fiscal year 2017 marketing plans for several organizations did not include 
end-of-year evaluation reports for all their marketing methods as required. ‘This 
amounted to 7 percent of the total marketing methods employed by regions and 
CVBs during fiscal year 2017. Out of 263 marketing method evaluation reports 
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reviewed, we identified 16 marketing methods without the required evaluation report, 
totaling over $75,000 that did not have the required evaluation report. For example, 
one organization received $15,000 to attend a tourism-related trade show, with no 
evaluation of what would constitute success in promoting tourism in that region. 
The number of evaluation reports reviewed is less than the research methodologies 
reviewed above, because if an organization uses solely non-bed tax funding to complete 
a marketing methodology, it is not required to submit an evaluation report for that 
methodology. Evaluation reports are only required of bed tax funded methodologies. 
If marketing methods are not evaluated, there is limited accountability regarding the 
use of public bed tax funding, making it difficult to gauge the success of marketing 
plans and overall marketing efforts by regions and CVBs. For example, there were 
instances of organizations not evaluating methods such as publicity, outreach, and 
marketing personnel. 


The Department Should Clarify and Enforce 
Research and Evaluation Requirements 


Presently, department and TAC regulations and procedures outline requirements for 
researching marketing methods before implementation, as well as the need to evaluate 
marketing methods after implementation. However, the research and evaluation 
requirements do not clearly define what is expected for organizations to submit for 
either the research or evaluation component for each proposed marketing method. 
The department’s regulations and procedures for marketing research and evaluation 
lack clarity for the expectations tourism regions and CVBs must follow. In addition, 
our work also found the department does not consistently require regions and CVBs 
follow the research and evaluation requirements that currently exist. For example, 
as noted, our review found that regions and CVBs did not consistently include any 
information regarding researching or evaluating marketing methods. ‘The department 
tracks tourism region and CVB evaluation reports via a marketing plan review master 
workbook. This workbook acts as a checklist to ensure evaluation reports have been 
submitted as required, with department staff contacting organizations which have 
omitted reports or need to provide clarification. However, we found mixed results in 
the department’s use of this workbook. The department does not routinely follow up 


with organizations to ensure missing evaluation reports are submitted as required. 


a 


RECOMMENDATION #3 


We recommend the Department of Commerce work in conjunction with the 
Tourism Advisory Council to improve its oversight of Tourism Regions and 
Convention & Visitor Bureaus by: 


A. Revising and clarifying regulations and procedures for marketing 
methodology research and evaluation requirements. 


B. Enforcing regulations and procedures, including ensuring bed tax 
marketing methodologies have required marketing methodology 
research and evaluation requirements. 


TL 


Tourism Spending Is Consistently Not Documented 


We noted the department inconsistently documents activities related to the distribution 
of tourism-related funding. This includes funding related to tourism grants and 
various financial reporting processes for tourism regions and CVBs. Documentation 
is an important tool in tracking and justifying how these funds are spent. A lack 
of documentation limits the accountability over how spending decisions are made. 
Based on current and past audit work, we determined there was a pattern within the 
department regarding inconsistent documentation for tourism-related activities. 


Pilot Grant Program Created Without 
Clear Guidelines or Support 


The department created a pilot grant program in fiscal year 2017 for tourism trade 
show assistance grants, which were funded as tourism grants. This pilot program was 
modeled after a manufacturing business trade show assistance program, offered by 
the department’s Office of Trade and International Relations. This pilot program 
was rolled out without adopting new forms or guidelines related to tourism-related 
goals; the forms and guidelines used were designed for manufacturing businesses 
by the Office of Trade and International Relations. As a result, three grants were 
awarded under unclear conditions as part of this pilot program. Additionally, all three 
grantees were deemed to have qualified for incentives, resulting in grant funding 
bonuses delivered under unclear circumstances. For example, two of the grantees 
each received a $200 bonus for being registered with “Visit Montana,” although the 
reimbursement form stipulated this bonus is available to grantees who participate in 
“Made in Montana” instead. A third grantee received a $500 bonus, supposedly for 
attending a trade show outside of the United States. However, the trade show attended 
was in Washington, D.C. Department staff indicated if a trade show has 10 percent 
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or more of attending companies from outside the United States, they consider it an 
international trade show per federal guidelines, and said this trade show met that 
threshold. However, the manner under which these grants were reviewed and awarded 
was not documented by the department. 


In all three grantees’ reimbursement forms, handwritten adjustments were made to 
each form to allow for the above reimbursements to take place. These adjustments 
ran contrary to the department's own Trade Show Assistance Program Guidelines. 
The department created an offshoot pilot program of the established manufacturing 
business trade show assistance program, without making any changes to the application 
or reimbursement form. The result was the appearance of grants and incentives being 
awarded seemingly arbitrarily and without any support. In all, nearly $7,000 was used 
from the regular tourism grant program funding pool for these grants. According to 
department staff, the pilot program was discontinued after a single grant cycle. 


Region and CVB Online Financial 
Reporting Missing or Unclear 


In reviewing region and CVB online financial reporting processes, we found both 
regions and CVBs did not consistently provide updated financial information as 
required. In fiscal year 2017, none of the 24 regions and CVBs submitted a required 
updated budget chart showing a percentage breakdown comparing the budget versus 
actual expenditures for each marketing method. This information is required per the 
department’s regulations and procedures as part of the evaluation reports submitted 
when marketing plans are completed. Similarly, not all regions and CVBs provided 
updated budget information comparing estimated and actual expenditures, whereas 
other organizations did do this. Current regulations require that regions and CVBs 
provide an updated budget comparing estimated budget amounts versus actual 
expenditures for each marketing method. In addition, some organizations had provided 
budget updates which contradicted the information in their evaluation reports. 
For example, one CVB noted no spending over the year as planned for a publicity 
marketing method. However, this CVB’s budget information indicated $2,000 was 
spent on the publicity marketing method, calling into question the accuracy of what 
is provided by organizations or submitted in evaluation reports. After we discussed 
this concern with department staff, the department directed regions and CVBs to 
make online updates to required financial reporting process information. According 
to department management, in some circumstances they have not clearly defined or 


enforced documentation requirements. 


Documentation Is a Standard Management Practice 


Documentation is a part of any effective internal control system. State policy outlines 
the need for an organization’s management to establish and maintain internal controls 
through a coordinated set of policies and procedures used by both managers and 
line workers to ensure effective, efficient, and lawful operations of the organization. 
Internal controls ensure accountability within an organization. There are several main 
components of internal controls, including the need for an organization to identify 
and exchange relevant information. Without this information and corresponding 
documentation, an organization will struggle to maintain effective internal controls. 
Over the course of our work, we noted inconsistent documentation within the 
department was common due to the department not prioritizing documenting 
all relevant information or decisions. When reviewing the fiscal year 2017 tourism 
grants, there were no procedures guiding documentation expectations. Similarly, the 
regulations and procedures guiding regions and CVBs lack detail on how to document 


required information. 
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RECOMMENDATION #4 


We recommend the Department of Commerce: 


A. Prioritize documentation regarding awarding and issuing tourism 
grants and required financial reporting processes for Tourism Regions 
and Convention & Visitor Bureaus by clearly defining documentation 
requirements in policies and procedures. 


B. Require documentation of tourism-related activities managed by the 
department, including tourism grants and required financial reporting 
processes for Tourism Regions and Convention & Visitor Bureaus. 
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October 11, 2019 


SET (MEN JECT 
Angus Maciver RECEIVE Ss) 
Legislative Auditor 

Montana State Capitol, Rm. 160 OCT 15 2018 

PO Box 201705 LEM ATIVE AUST ON, 


Helena MT 59620-1705 


Re: Performance Audit 
Tourism Marketing and Promotion 


Dear Auditor Maciver, 


The Department of Commerce has reviewed the October 2019 Legislative Audit Division’s report from the 
performance audit of our tourism and marketing promotion efforts. Our response to the 
recommendations are as follows: 


Recommendation #1 


We recommend the Department of Commerce develop and implement procedures to document and 
support grant award and nonaward decisions for tourism grants based on established grant award criteria. 


Response #1 


The Department concurs. Consistency and transparency are important. Since receiving feedback during 
the audit review, we have made improvements to the grant review and documentation process. We will 
continue to improve on consistent and transparent documentation, including an enhanced online grant 
application that provides explanations, helpful links, and the ability for the applicant to upload documents. 
Further, decision-making documents including team scoring matrices, comments, and scoring summaries 
will be retained in an electronic system for three years. 


Recommendation #2 
We recommend the Department of Commerce develop and implement: 
A. A process with defined criteria to measurably evaluate the impact of tourism grants or 
document why an individual grant’s activities are not measurable, and 


B. Clear guidelines for grantees to follow and a structured template to use as part of a final 
reporting process. 
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Response #2 


The Department concurs. Having measurable results and data to help guide future decision making is an 
important component of strategically investing in the tourism industry. When possible, grant recipients 
will be required to utilize SMART criteria to report on grant impact as part of the recipient’s final report. 
When the impact of a grant is not measurable because of time constraints, is cost prohibitive, or lacks 
substantive measurable data, we will document why impact cannot be measured. A process for 
evaluating grant impact will be in place for the 2020 tourism grant cycle. 


As to guidelines and structured templates, we have made improvements since receiving feedback during 
the audit review. In advance of the 2020 tourism grant cycle, we will create standardized templates for 
grant recipients, such as templates for detailed budgets, request for funds, and final reports. Further, we 
will ensure guidelines and templates are in place for all tourism grants, including pilot grant programs as 
they occur. 


Recommendation #3 


We recommend the Department of Commerce work in conjunction with the Tourism Advisory Council to 
improve its oversight of Tourism Regions and Convention & Visitor Bureaus by: 


A. Revising and clarifying regulations and procedures for marketing methodology research and 
evaluation requirements. 

B. Enforcing regulations and procedures, including ensuring bed tax marketing methodologies 
have required marketing methodology research and evaluation requirements. 


Response #3 


The Department concurs. Oversight of Tourism Regions and Convention & Visitor Bureaus occurs in 
conjunction with the Tourism Advisory Council (TAC) and their guidelines and regulations. TAC and the 
Department will review and clarify language related to regulations and procedures for marketing 
methodology research and evaluation requirements. Clarifying language and review forms will be 
presented to TAC for approval in February of 2020 and will be implemented in May of 2020. 


In addition, the Department will immediately begin reviewing its current enforcement mechanisms 
related to bed tax marketing methodologies and will make improvements to ensure regulations and 
procedures are enforced. The Department will perform quality assurance to ensure records are complete 
and that we do not have an issue with missing or incomplete information. Improved enforcement 
mechanisms will be in place by spring of 2020. 


Recommendation #4 
We recommend the Department of Commerce: 
A. Prioritize documentation regarding awarding and issuing tourism grants and required 


financial reporting processes for Tourism Regions and Convention & Visitor Bureaus by clearly 
defining documentation requirements in policies and procedures. 
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B. Require documentation of tourism-related activities managed by the department, including 
tourism grants and required financial reporting processes for Tourism Regions and Convention 
& Visitor Bureaus. 


Response #4 


The Department concurs. We agree that accountability and transparency are essential as stewards of 
state resources. However, the department notes the tourism grant component of Recommendation #4 
was already addressed by the Legislative Audit Division in Recommendation #1 and responded to by the 
Department in Response #1. Similarly, financial reporting on marketing methods was already addressed 
as part of Recommendation #3 and responded to in Response 3. However, we will work in conjunction 
with TAC to clarify documentation requirements and prioritize documentation regarding financial 
reporting processes for Tourism Regions and Convention & Visitor Bureaus. 


The Department thanks you for the diligent efforts of the Legislative Audit Division and for collaborating 
with us during the exit conference. 


‘Sincerely, 


Tara Rice, Director 
Montana Department of Commerce 
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